In search of significant differences.

Consider reading the following: "After lengthy study, no statistically significant differences were found between male and female subjects, however, the trend shows ...’" or "Variables 1 and 2 were significantly different at the 0.01 level, but variables 3 and 4 were different at only the 0.05 level; however, variables 5 and 6 were different at the 0.001 level." Such statements indicate a basic misunderstanding of the purpose of statistical testing, and often are prompted by a desire of investigators to find positive differences between groups. The notion that research must demonstrate positive differences between groups in order to be good is mistaken. Careful research should seek to explore reality rather than prove a bias. The following discussion will differentiate briefly between descriptive and inferential statistics in order to consider the purpose of statistical testing. Two guidelines for investigators are presented. When statistical tests are to be performed, the level of probability should be determined prior to the analysis of that data. More important than statistical testing is the interpretation of findings to determine if statistically significant differences are meaningful within the context of the research questions being asked.